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ABSTRACT 

. ^, This paper, presented at the Fifth Annual Conference 

Of the Australian Science Education Research Association in 1974, 
discusses the various strategies for analyzing the data gathered to 
explore the effect of specific school variables on educational 
achievement. The discussion centers around the International 
Association for the Evaluation of Educational Achievement (lEA) 
strategy of analysis, the use of variance estimates, the use of 
variation estimates, and the treatment of compound variables. The 
author contends that none of the strategies is completely 
satisfactory and calls for further discussion and experimentation. 



♦ Documents acquired by ERIC include many informal unpublished * 

♦ materials not available from other sources, ERIC makes every effort * 

♦ to obtain the best copy available, nevertheless, items of marginal * 

♦ reproducibility are often encountered and this affects the quality ♦ 

♦ of the microfiche and hardcopy reproductions ERIC makes available ♦ 

♦ via the ERIC Document Reproduction Service (EDRS) , EDRS is not * 

♦ responsible for the quality of the original document. Reproductions * 

♦ supplied by EDRS are the best that can be made from the original, ♦ 





i 



1974:2 



CO 
CO 

o 




The effects of the conditions of learning 
in the schools on educational achieveoent 

John P. Keeves 



9 



PERMlSSrON TO REPnOOUCE TMiS COPY 
RtGMTEO MATEPlAl mas SEEN GRANTED BY 

TO ERiC AND 0RGAN7ATI0NS U^ ERATJNG 
UNDER AGREEMENTS WJtH Tt-'t NATtONAl (N 
STITUTE OF EDUCATION puRTmER REPRO 
OUCTION OUTSrOE TmE fRrO ..>oTEM RE- 
OUiRES PERMISSION Of IHf '.OPYRIGmT 
OWNER 



U S. DEPAKTMENTOP HEALTH, 
EDUCATION ^WELFARE 
NATIONALINSTITUTEOF 
EDUCATION 
THIS DOCUMENT HAS BEEN REPRO 
DUCED EXACTLY AS RECEIVED FfROM 
THE PERSON OR ORGANIZATION ORIGIN 
ATlSG IT POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRE 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION POSITION OR POLICY 



(so 

o 

o 

\ 



lEA (International Association for the Evaluation of Educational Achievement) 
is an international educational research organization. 
ACER (Australian Council for Educational Research) is the National Centre 
for lEA activities in Australia. 




The effects of the conditions of learning 
in the schools on educational achievement 



A discussion of the debate between J.S, Coleman and G»F» Peaker 
at the Harvard-IEA Conference on Educational Achievement 
at Harvard University, November 1973 



lEA (Australia) Report 1974; 2 



John ?• Keeves 
Associate Director, ACER 



Australian Council for Educational Research 
PO Box 210, Hawthorn, Victoria 3122 



Copyright (c) ACER, July 1974 



ISSN 0310-.558X 



An edited version of a paper presented at the Fifth Annual Conference 
of the Australian Science. Education Research Association, Monash 
University, Mclboame, 18-20 May 1974* 



ERLC 



5 



Through the night of doubt and sorrow 
Onward goes the pilgrim band 
Singing songs of expectation 
Marching to the Promised Land. 



Translated from the Danish by B.S. Ingemann and quoted by 
Gilbert Peaker during the period when computer programs were 
being prepared and some of the issues discussed in this 
report were being argued out at the lEA data processing unit 
in Stockholm, in Winter 1971-72. 
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The International Studies In Evaluation are huge and complex cross-national 

investigations into the educational achievement of students in general at 

three stages of schooling, the 10-year-old, the 14-year-old and the 

terminal secondary school levels, and in seven subject areas: Mathematics, 

Science, Reading, Literature, Civic Education, English as a foreign language, 

and French as a foreign language. In November 1973, at Harvard University, 

a group of educators, sociologists, economists, and statisticians met the 

team of research workers who had been engaged in these studies to discuss 

the meaning and relevance of the lEA findings for educational policy and 

practice and to consider their implications for future cross-national 

educational and social research. One of the papers presented at the 

Harvard-IEA Conference was by James S. Coleman of the University of Chicago, 

on the "Effects of School on Learning: The lEA Findings". In this 

paper Coleman stated: 

In arguing for measures other than the added exi)lained variance 
or unique variance, I must point out that I have until very 
recently used these measures in all my work, and in Equality of 
Educational Opportunity , we used then. Although I still believe 
these measures are superior for the present purpose to others 
currently in use, I think they are not as good as the alternatives 
I am proposing here. (Coleman, 1973 a : 18) . 

Peaker, who had helped to guide the lEA an.ilyses during the first phase of 
the investigations into Mathematics and after a period of some uncertainty 
had directed the analyses during the second phase of the studies in the six 
subject areas, replied drawing attention to inconsistencies in Coleman's 
paper (Peaker, 1973a). Coleman hurriedly modified his original paper and 
some of the procedures he was proposing, but did not change his ba&ic approach 
(Coleman, 1973b). Peaker again replied suggesting that the quantity Coleman 
was proposing as a "measure of equality of educational opportunity" was very 
dubious while the "measure of the effectb of school variables on achievement, 
that arc ;.iidepcndc;n': of fanraZy background and school type" had indeed been 

ese studies have been undertaken by the International Association for 
cKJC the Evaluation of Educational Achievement (lEA) . 



presented in the lEA reports in the squared form (Peaker, 1973b). In the 
discussions that followed at the Harvard-IEA Conference, the issues were not 
clarified, with neither Coleman nor Peakur yielding ground. The purpose of 
this paper is to attempt to raise some of the issues involved, to place them 
in perspective and to suggest which measures and procedures of analysis are 
likely to be most useful l.n future studies • 

Over the past decade several major investigations, in addition to the 
lEA studies, have bee undertaken, which have presented evidence on the 
magnitudes of school effects for student achievement, Peaker has reported 
the findings of the Flowden National Survey (Peaker, 1967a) and its follow-up 
study (Peaker, 1971), Coleman was the main author of the report of the 
Equality of Educational Opportunity Survey (Coleman, 1966). In general, 
chase studies, together with the lEA work, reveal that school variables, as 
assessed in these inquiries, show little effect on student achievement. The 
analyses carried out relate the variance existing in the independent variables 
to that existing in the dependent variable, and the smallness of a reported 
effect may arise from homogeneity associated with the independent variable 
(Walker, 1973), Nevertheless, the lEA studies do indicate that school effects 
are more substantial for the learning of school based subjects such as Foreign 
Languages and Science and are considerably smaller for the learning of such 
subjects as Reading (Comber and Keeves, 1973; Thorndike, 1973; Carroll, 1974; 
Lewis, 1974). These findings are disappointing for those who argue the need 
to improve the quality of schooling and who attempt to identify factors which 
contribute to disparities in educational achievement among students in 
different school and societal situations. If the important school factors 
could be determined and the magnitudes of their effects assessed, then a 
case could be argued for the allocation of resources to improve the conditions 
of schooling for those students whose level of achievement in education is 
lower than might be expected, 
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Following the publication of the reporc, Eq uality of Educational 
Opportunity with its evidence suggesting that there wad relatively little 
that schools could do to raise the level of performance of students in 
the United States, attempts were made to reanalyse the data and to 
present the findings in alternative ways that would assess more accurately 
the contributions of the schools. One important such investigation, 
A Study of Our Natlonb Schools bv Mayeske (1969), suggested that alternative 
techniques for the analysis of the data yielded somewhat different findings. 
This procedure showed that the school shared its effects in part with the 
home, but it was not very successful in identifying school factors that 
contributed markedly to student achievement. Others have criticized the 
techniques used by Coleman and his co-workers (for example, Bcwles and 
Levin, 1968; Dyer, 1969), Consequently the recommendations made by 
Coleman for superior measures and methods of analysis are clearly based on 
a sound knowledge ol the complexity of the task and an awareness of the need 
for the use of procedures that are free, as far as is possible, from the 
criticisms raised. 

While prolonged consideration of methodological issues may sometimes 
appear an unrev/arding and barren task, it is nevertheless an important one. 
Guidance is needed for attempts to reduce the inequalities known to exist 
in educational opportunity. Programs of compensatory education are 
required and wh. re possible they should be based on research evidence. 
Consequently, it is desirable that the analytical procedures employed to 
examine research data to provide this evidence should be powerful and 
unambiguous • 
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The lEA Strategy of Analysis 

The lEA invvistigations carried out so far have been cross-sectional 

studies in which inferences have been made about the influence of home 

and school factors from evidence collected at one point in time. 

Longitudinal studies (for example, Peaker, 1971; Keeves, 1972) are 

free from some of the criticisms levelled at cross-sectional inquiries, 

but they are more time-consuming, costly, and more technically complex. 

Peaker (1973 b : 4) has pointed out that in the absence of pre-test data 

assessing student initial achievement, it is necessary to simulate the 

pre-test by using information about the student's home circumstances and 

the type of school he attends or the type of course or program he is 

engaged in. Thus in the lEA analyses the variables examined for their 

contributions to educational achievement were grouped into four blocks 

according to their type and the sequence in which they were believed to 

effect student achievement. The blocks were defined roughly as follows: 

Block 1. Home and Student Background, comprising Home Circumstances, 
Age, and Sex; 

Block 2. Type of School and Type of Course or Program, where there 
was sufficient differentiation within a sample for such 
variables to be formed; 

Block 3* Learning Conditions in the School^and Grade; 

Block 4. "Kindred" Variables, so called because they were contemporaneous 
with the criterion in time, including such measures as school 
motivation and interest in the subject. 

Furthermore there was the criterion variable which was a measure of 
student achievement. 

A few comments are needed about some of these measures because 
misunderstandings have occurred. First, the Block 4 variables cannot be 
considered rightly as determinants of achievement, since they are also 
influenced by achievement. In the development of a recursive causal model 
it is necessary that they should be excluded from consideration. Secondly, 
the Block 3 variables are those of greatest interest. In general, they 
^ represent factors which can be modified to influence student achievement. 
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In an age sample the variable Grade, while indicating exposure to learning, 
way be related to the ability of the student or to promotion policies in 
schools of different types. Thirdly, the Block 2 variables concerned with 
the Type of School the student attends and the Type of Course he is 
following are related in some countries to the performance of the student 
prior to the time he comes under survey. However, a careful examination 
of how these variables are defined shows that in some countries these, 
measures are clearly linked with the home background of the student, sex, 
the region in which the school is located and the students' perceptions of 
the courses they are taking at school. Many of these factors are not 
necessarily related to the prior performance of the student. Coleman 
rightly points out using Peaker's yacht handicap analogy (see Comber and 
Keeves, 1973: 194-6) that yachts with the greatest handicap also have 
better crews, or in school terms the schools holding the more able students 
tend to provide better conditions for learning (Coleman, 1973a: 10). In 
addition, schools for students from more favoured homes, or from more 
affluent regions, or for boys, may provide better learning conditions than 
do schools for students less fortunately placed. Fourthly, the variables 
in Block 1 cannot be seen solely as measures of the home circumstances of 
the students, since sex and age are also included in this block. These 
variables are, however, fixed and not subject to change. 

In order to examine the interrelations between the variables in the 
first three blocks and student achievement and in the absence of information 
on the student's initial level of skill, it is helpful to develop a simple 
causal path model. Coleman (1973a: 11) makes clear that he is "not an 
advocate of path analysis", but he does use such a path diagram very 
effectively to clarify certain issues in the analysis of the data. In 
a cross-sectional study it may be argued that the Block 1 variables. Home 
and Student Background, influence the Type of School the student attends 
and the Type of Course or Program that he follows (The Block 2 variables) . 
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Furthermore, both the Block 1 and Block 2 variables influence the 
Learning Conditions in the School (Block 3 variables) and the 
variables in all three blocks directly and indirectly influence student 
achievement. The network of interrelationships among the three blocks 
of variables and Achievement may be represented by the causal path model 
shown in Figure 1. 

The estimation of the magnitudes of the interrelations among the blocks 
of variables and their components in such a causal model is undertaken by 
regression analysis. Thus the use of the causal path model is consistent 
with the regression analysis procedures used in most of the studies that 
have been referred to. The same model was used in the lEA analyses for 
the examination of data where both schools and students were the units 
of analysis, although the variables included within the blocks differed 
slightly in their composition from subject to subject, from population 
to population, and from the between school analyses to the between student 
analyses. Underlying this causal model is a line of reasoning that 
earlier events in the life of the child influence later events, and the 
path diagram makes explicit the causal sequence employed to study school 
effects and their dependence on the home and student background measures. 
While it may be possible to challenge some of the measures included in 
the different blocks, it is difficult to quarrel with this causal model 
and Coleman (1973a: 11) does not do so, nor does he question the use of 
regression procedures. The debate between Peaker and Colaman is within 
the framework of regression analysis and is directed towards the choice 
of statistical measures to express the relationships assumed to exist and 
the techniques employed to separate out and assess hypothesized effects. 
The discussion focusses on two procedures (1) the use of variance estimates 
and (2) accounting for variation in scores. 
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A Path Diagram for the Causal Model used in the lEA Studies. 

denotes the Home and Student Background Variable 
denotes the Type of School or Program Variable 
denotes the Learning Condition Variable 
denotes the Achievement Outcomes 

denotes the residual variables for Type of School or Program 
denotes the residual variables for Learning Conditions 
denotes the residual variables for Achievement 
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The Use of Variance Estimates 

The major educational studies carried out by Peaker, Coleman and others 
have examined the effects of individual factors or collections of factors 
on student achievement using variance estimates in which the contribution 
of the variable or collection of variables in accounting for the variance 
of the achievement test scores is assessed. Peaker (1967a) in the 
report of the Plowden National Survey estimated the contributions of 
individual variables to explaining the variance of the achievement test 
scores using the product of the zero-order correlation coefficient (r) 
with the standardized partial regression coefficient (b) (Guilford, 1956). 
Darlington (1968) has criticized the tse of this procedure since for 
suppressor variables the product is negative and a negative contribution 
to explained variance does not appear to be meaningful (see Ward, 1969; 
Duncan, 1970). 

An alternative measure that is clearly appropriate to use where the 
predictor variables are orthogonal or where all variables can be ordered 
in a clearly defined temporal sequence is the square of the semi-partial 
correlation coefficient at each step in the analysis. This measure is 
the variance added at each step of the regression analysis and was used 
in the report of the Survey of Equality of Educational Opportunity (Coleman, 
1966). 

In the reanalysis of this data reported by Mayeske (1969), the estimates 
of variance accounted for were separated into unique and joint contributions 
follo\^ .ng a procedure which has been stated more fully by Wisler (1969) 
and Mood (1971). This technique was advanced earlier by Newton and 
Spurrell (1967) and was employed also by Peaker (1971) in the report of 
the Plowden Follow-up study. Peaker (1967b) noted that under certain 
circumstances the joint contributions to variance explained could be 
. negative, a shortcoming of this measure that had also been recognized 
by Mood (1971). The conditions under which negative joint effects arise 
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have been considered in detail by Beaton (1973). They are linked with 

effects that are similar in kind to suppressor relationships. The 

unique contribution to the variance accounted for is the square of the 

standardized partial regression coefficient (b) multiplied by the 

standard deviation of the residual (s) at the penultimate stage. 

This measure (b s , sometimes recorded as b / , where c is the 

c 

term ii: the diagonal of the matrix of the regression coefficients at 
the ultimate stage) is a lower bound for the variance contribution of 
the variable and provides some assessment of its separate contribution 
and thus of its relative importance. 

The use of variance measures permits the erfects of individual 
variables to be combined by simple addition or subtraction in 
order to obtain estimates of the unique and joint contributions of blocks 
of variables and where required to obtain the independent and total effects 
of the collections of variables in the blocks. 

In general in the lEA reports variance estimates hav; been used. 
It was clearly necessary to combine the effects of variables 
particularly in Block 3, where the contributions of individual variables 
assessing the effects of the different learning conditions in the schools 
were small. The increments in variance associated with the inclusion 
of each block of variables in the regression equation have been presented 
in the reports to provide evidence on the relative importance of the 
different factors incorporated within each block. Furthermore, in the 
Science report (Comber and Keeves, 1973) it was argued that there was 
substantial overlap between the Block 2 variables, Type of School and 
Type of Course or Program, and the Block 3 variables. Learning Conditions 
in the Schools. Consequently, to assess accurately the school effects it 
was necessary to combine the variance estimates for these two blocks and 
Q to express them in terms of unique and joint contributions. 
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It is important to note that cases arise in the lEA reports, where 
the joint contributions of the variables in Blocks 2 and 3 are negative, 
casting some doubt on the Newton and Spurrell method of assessing the 
effects of both individual variables and blocks of variables since a term 
involving a negative contribution to the total variance gives rise to 
problems of interpretation. A further shortcoming of the use of variance 
estimates of the effects of variables is that insufficient attention is 
sometimes given to the direction in which factors are working. Where 
suppressor relationships are involved they are not always exposed for 
scrutiny. Some discussants of the lEA results have been confused by effects 
of this kind which occur in the evidence presented (for example Bloom, 1973) 
and as a result have misinterpreted the direction in which factors are 
operating. Not only are such suppressor relationships sometimes difficult 
to interpret, but they may not always give proper insight if weights are 
taken at face value. The problem arises from the interdependence among the 
variables in the regression equation in such a situation, and the variables 
should not be interpreted as if they were independent factors (see Van de Geer 
1971s 126-127). 

The Use of Variation Estimates 

Coleman (1973a) argues the case for the use of alternative measures that 
have a direct operational meaning in terms of predicting differences in 
scores. Variance measures involve the squares of the differences, 
variation measures are based on the unsquared differences in the scores 
and therefore also encompass the notion of direction as well as magnitude. 
The emphasis is on the capacity of a variable or a block of variables to 
predict variation in the achievement test scores rather than accounting 
for the variance of the test scores. Coleman directs attention to 
three such measures which respectively assess the contributions to 
O variation of: 




1. The effects of Learning Condition variables through wliich Home 
Background and Type of School variables are acting. 

2. The effects of Learning Condition variables which are independent 
of the effects of Home Background and Type of School variables. 

3. The total direct effect of the Learning Condition variables. 

If we consider the Home, School and Learning Condition blocks of variables 
as specified in the causal model presented in Figure 1, the third measure 
is given by the standardized partial regression coefficient for Achievement 
regressed on the Learning Condition variables controlled for the Home and 
Student Background variables and the Type of School or Type of Program 
variables (b^^ 21^ ' This quantity is reported for individual variables in 
two of the reports (Purves, 1973; Thorndike, 1973) but in the Science 
report (Comber and Keeves, 1973) it is only presented for those variables 
where the pattern of results suggests that a factor is acting significantly 
and consistently across countries for a given population. This quantity 
was not, hov/ever, calculated or reported for the learning condition 
variables as a block. 

The second measure as Coleman (1973b) showed is given by ^^2.12* ^3.12 

or b,^ v/ 1 - R^o -lo* This measure is the square root of the unique 
43.12 V 3.1^ 

contribution of a block of variables, and some evidence on this measure 
was presented in square form in the reports. A more detailed treatment 
will be presented in square form in the technical report of the lEA studies 
(Peaker, 1974). 

The first measure is given by the product ^/^2.2V ^3.21' ^^^^^ ^3.21 
the standardized regression coefficient for Block 3 variables regressed 
on a compound variable for Block 2 and Block 1. 

Coleman (1973a) presents some computational formulae for calculating 
these coefficients when only indices for individual variables and multiple 
correlation coefficients have been provided by computer programs. It 
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would appear that what Coleman is really suggesting is equivalent to an 
examination of relationships between compound, but unobserved, variables 
for the hypothesized blocks using path analysis techniques. The 
calculation of direct and indirect causal path effects for the causal 
path model presented in Figure 1 would appear to be largely equivalent 
to the calculation of the measures he has proposed. He has suggested in 
passing that regression weights could be employed in the formation of a 
new compound variable (Coleman, 1973a: 27), but since this is not the 
only way of proceeding, this problem will be considered separately later 
in this paper. 

Although variance estimates allow measures of the effects of a set 
of variables to be obtained readily from data on the effects of single 
variables, Coleman argues for the use of variation estimates on the 
grounds that these latter measures are more easily interpreted having an 
explicit meaning in terms of score differences. Moreover, since the 
results are not reported in the square form, they are not only directly 
comparable from one set of standardized variables to another, but they also 
lead to greater optimism about the relative size of the effects of the 
school variables. The thrust of Coleman's papers (1973a, 1973b) is the 
desirability of using variation measures, regression weights and path 
coefficients or their equivalent, instead of variance measures to examine 
the relative effects of individual variables or collections of variables. 
A Measure of Equality of Educational Opportunity 

Coleman (1973a: 33-34) has proposed, using measures involving estimates of 
the effects of blocks of variables on the variation in the achievement test 
scores, an index of "equality of educational opportunity". Two of the 
three measures discussed above are employed. The second measure assessing 
the effects of learning condition variables which are independent of the 
•effects of home and school type may be regarded as the actual influence 
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of the schools in equalizing educational opportunity. The third measure 
gives the total direct effect of the learning condition variables and may 
be thought of as the capacity that the schools have for equalizing 
educational opportunity if learning conditions were distributed without 
regard to home and student backgrounds and school type. The proportion 
of the total direct effect of the schools which is distributed 
independently of home and school resources can be considered as a 
measure of equality of educational opportunity, Peaker (1973b: 5) 
draws attention to the fact that this ratio is: 

^ Effect of schools independent of home and school type 
Total direct effect of the schools 

^43,12 
^ ^3.12 

Further, since this ratio is equal to the residual standard deviation for 

the learning condition measures after inclusion of the home and school 

type measures in the regression equation Peaker suggests: 

To the writer, at any rate, it seems reasonable to conclude that 
while the case may be somewhat altered when surrogates are used 
for the pre-test it is not altered very much, and that the ratio 
does not tell us a great deal about equality of opportunity 
(Peaker, 1973b: 6). 

To support this view he uses data from The Plowden Children Four Years Later 
(Peaker, 1971) and argues that such a measure while superficially attractive 
may present problems of interpretation. In discussion neither Coleman 
nor Peaker yielded to the views of the other. It would appear necessary 
to test this ratio with data from a variety of sources to assess the 
usefulness of this measure of equality of educational opportunity. 



The Treatment of Compound Variables 

Some of the Issues associated with the combination of observed variables to 
form compound unobserved variables have been examined by Hauser and Goldberger 
(1971). Four approaches have been employed In recent studies for dealing 
with these problems; these use (1) principal component analysis, (2) restricted 
maximum likelihood factor analysis, (3) regression analysis, and (4) canonical 
analysis. An alternative approach has been advanced by Wold (1974) which 
uses non-linear Iterative partial least squares modelling (5). 

Principal Component Analysis 
1. In the discussions following the presentation of the paper by Coleman 
^ at the Harvard-IEA Conference the use of we-'.ghts derived from a 
principal comporent analysis was suggested for the formation of a 
compound variable. This strategy was used by Keeves (1972). A 
similar approach Involving the use of Varlmax Rotation and Factor 
Analysis had been used by Coleman (1966) and Peaker (1967a). These 
procedures are appropriate where the variables can be assumed to be 
measured without error, and where one or more unldlmenslonal 
unmeasured factors are associated with the measured variables. 
2* Restricted Maximum Likelihood Factor Analysis 

This procedure has been advanced by Joreskog (1970), and developed by 
Hauser and Goldberger (1971). It Involves the use of confirmatory 
factor analysis. It is appropriate where the variables are measured 
with error and a correction is applied for such errors. Since certain 
zero factor loadings are necessarily specified in advance, the unmeasured 
factors must be readily identified. This approach has been used 
recently by Williams (1973). 
3* Regression Analysis 

Tukey (undated) advanced the idea of using rosettes as carriers of 
regression in which the measured variables were combined using 
regression weights to form the unmeasured compound variable. The 
O regression weights can be calculated with all measured variables 
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with only those variables incorporated in^a specific compound being 
included. This procedure makes no correction for attenuation, nor 
does it assume that a unidimensional factor lies beneath the compound 
measure so formed. Furthermore it serves to maximize the relationship 
between the measured variables and the criterion in the formation of 
the compound. This procedure was employed in part in the lEA analyses, 
and Coleman (1973a) proposes an extension of this principle to form 
compound measures associated with the predictor blocks of variables. 

Since the reanalysis of data is tedious after each new compound 
variable is created Coleman (1973a Appendix) presents two procedures 
by which path coefficients can be calculated using data normally 
available from regression analysis results. 
Canonical Analysis 

Cooley and Lohnes (1974) have developed procedures by which analyses 
might be carried out in situations where several predictors are 
compounded into blocks and where several criterion measures are used. 
Traditionally, in canonical analysis the interpretation of the canonical 
factors has been based on the standardized weights, but Cooley and 
Lohnes (1971) adopted a procedure in which the canonical factor structure 
was defined by the correlations between the original observed variables 
and the derived canonical variables. This provides a more meaningful 
interpretation of the canonical factors, while the canonical weights 
tell the relative contributions of each original variable to the 
canonical factors. In addition, Cooley and Lohnes (1974) extended 
the use of the redundancy measure in order to examine the commonality 
between the blocks of variables and they illustrated the use of these 
procedures with several sets cf data. 
NIPALS (Non-iterative Partial Least Squares) Modelling 
In general terms, the NIPALS approach leads to the construction of 
non-linear models by linear methods. Following the specification of 
the model, with due regard to its intended use, the unknown parameters 



and latent variables are estimated using iterative ordinary least 
squares regression procedures. Wold (1974) claims several advantages for 
this approach, not the least of which is the economy of assumptions, 
that often make the NIPALS model more general than alternative models • 
As yet this approach does not appear to have been applied extensively 
in the analysis of educational and sociological data. 

Summary and Conclusions 

The results of the surveys conducted by lEA and by the United States Office 
of Education in the study of Equality of Educational Opportunity have 
indicated that to examine the effects of specific school variables using 
the data collected in surveys is at best a hazardous procedure. 
Experience suggests that it is more effective to examine clusters of 
related school variables since each individual variable has only a small 
effect. Several procedures have been suggested for combining together 
specific measures to form an unmeasured composite whose overall impact 
can be more accurately assessed. Which procedure is to be employed must 
depend on the nature of the specific measures being combined and the nature 
of the model which has been developed and is being examined with the data 
collected. However, in the TEA studies where a model was developed 
consisting of three blocks: Home and Student Background, Type of School 
or Program, Learning Conditions in the School, and with Achievement as 
the criterion variable, it would appear most appropriate to form compound 
measures based on regression weights. 

Nevertheless, it would appear to be appropriate for a series of 
analyses to be carried out on a set of data to explore the use of the first 
four techniques discussed above. Each no doubt has its strengths and its 
limitations and only by examining in detail the alternative procedures 
available can issues associated with the formation of unmeasured variables 
be resolved. The maximum likelihood approach and related techniques 
£P^C would, however, appear to be currently gracing in favour. The usefulness 



of NIPALS modelling should also be explored v^ith data from educational 
and sociological studies, since without illustrative analyses it is 
difficult to assess the value of this promising approach. 

The use of regression procedures in the analysis of the lEA data 
was unchallenged in this debate; what was questioned by Coleman was the 
most meaningful way in which relevant evidence could be extracted from 
the analyses. Earlier Cronbach (1957) advocated a convergence of the 
procedures used by the experimenter in psychology with those employed by 
the correlational psychologist in the examination of evidence in scientific 
psychology. Since then regression analysis, in particular, and canonical 
analysis have been so developed that they provide in part a convergence 
of techniques that is consistent with the merging of approaches which 
Cronbach sought. Nevertheless, there is still some lack of agreement 
among analysts as to whether variance estimates or regression weights and 
variation estimates should be employed in the presentation and interpretation 
of the findings. 

Those experimenters who have been schooled in the Fisherian tradition 
of analysis of variance would appear to prefer to express the evidence 
in terms of variance effects and increments to variance explained. 
However, econometricians , who have had greater experience witl path analysis 
techniques, advocate the use of path coefficients, regression vieights and 
their like. Coleman (1973a, 1973b) in his discussion of the analyses of 
the lEA data has proposed a shift from the use of variance measures to 
variation measures and regression weights although it should be noted he 
does not advocate the use of path analysis techniques. Peaker would seem 
to suggest that the difference in form between the square and the unsquared 
term is, largely trivial, and in interpreting the findings it is possible 
to shift from one form to the other as is most appropriate. Coleman 
argues that the virtue of the va. Action measures "is that the numbers 
have a direct operational meaning in terms of predicted differences in 
scores under particular hypothetical experiments" (Coleman, 1973b: 8) ami 
ERs£C that "the use of the squared pfithex than the unsquared form leads to an 



unwarranted pessimism about the size of the effects" (Coleman, 1973: 31). 
If space allows in the presentation of results from studies in the 
immediate future it would appear desirable to use both forms and to 
draw inferences from both. Peaker (1971) has used such an approach very 
successfully in reporting the results of the study of The Plowden Children 
Four Years Later . But it is only by the use of both sets of measures in 
ways in which comparisons are possible, that the question of their relative 
usefulness will be resolved. 

A further innovation suggested by Coleman Is an index of equality 
of educational opportunity. The development of a measure to assess this 
concept could serve an extremely useful purpose in both cross-national 
and cross-regional studies. However, it is questionable whether the 
quantity that Coleman is proposing when denuded of its verbal embellishments 
does actually provide such a measure. 

lEA owes a substantial debt to both authors. Gilbert Peaker has 
for pearly a decade steered the lEA analyses between the Scylla of 
superficiality and the Charybdis of cost in extracting every possible 
result from a plethora of data. In th. dark days of 1965 he suggested an 
alternative approach for the analysis of the data in the Mathematics Study 
and again in late 1971 and early 1972, when other helpers had failed, 
he guided day by day from a distance of many miles the analyses which were 
carried out. On the other hand James Coleman visited the Stockholm data 
processing unit only for a few brief hours and from the wisdom and experience 
gained in examining the data collected in the Equality of Educational 
Opportunity Survey he gave advice on two critical problems that greatly 
assisted the work that followed and the results that have been presented. 

The debate between Peaker and Coleman fit the Harvard-IEA Conference 
could well advance very substantially the strategies of analysis employed 
in examinins the data gathered not only in future large scale surveys but 
also in large scale experimental studies. 
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